A structural equation modeling approach for examining position effects in large-scale assessments

Background Large-scale assessments in education are typically administered using multiple test forms or booklets in which the same items are presented in different positions or locations within the forms. The main purpose of this practice is to improve test security by reducing the possibility of cheating among test takers (Debeer and Janssen 2013). This practice also helps test developers administer a greater number of field-test items embedded within multiple test forms. Although this is an effective practice for ensuring the integrity of the assessment, it may result in context effects—such as an item position effect—that can unwittingly influence the estimation of item parameters and the latent trait (Bulut 2015; Hohensinn et al. 2011). For example, test takers may experience either increasing item difficulty at the end of the test due to fatigue or decreasing item difficulty due to test-wiseness as they become more familiar with the content (Hohensinn et al. 2008). It is often assumed that position effects are the same for all test takers or for all items and thus do not have a substantial impact on item difficulty or test scores (Hahne 2008). Abstract